Refactor argument parsing and passing to a json-based approach instead of code injection#762
Open
rcannood wants to merge 150 commits into
Open
Refactor argument parsing and passing to a json-based approach instead of code injection#762rcannood wants to merge 150 commits into
rcannood wants to merge 150 commits into
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
10 tasks
* add improved json parser * improve speed
Member
|
Updates to mitigate breaking changes:
|
Member
This is nicely covered, including tests!
I don't see how this is covered?
I don't see where this is covered either? |
the old images are on old debian versions that aren't maintained any longer, resulting in unmaintained repositories where we can't update or install jq any longer
Collaborator
|
2 issues remain in tests:
|
Use Scala 3.6 as Scala 3.8 no longer supports Java 11 Start Scala/Bloop engine as a bunch of downloads need to happen and needs some time to wake up preventing possible stuck servers down the road
hcannoodt
approved these changes
Mar 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
This PR refactors Viash to use JSON-based parameter passing instead of direct code injection. This makes argument value handling more robust, easier to debug, and correctly handles special characters like backticks, quotes, dollar signs, and newlines.
Related issue(s)
Fixes #375, #619, #705, #763, #821, #840
Changes
io.viash.languagespackage: Language trait with implementations for Bash, Python, R, JavaScript, Scala, C#, and Nextflowparams.jsonand parsed at runtime using language-specific JSON parsers (no external dependencies required)Breaking changes
Multiple-value input arguments are now arrays
Before:
After:
JSON Structure
{ "par": { "input": "/path/to/input.txt", "whole_number": 42, "real_number": 3.14, "multiple": ["a", "b", "c"] }, "meta": { "name": "component_name", "resources_dir": "/path/to/resources", "executable": "/path/to/executable", "config": "/path/to/.config.vsh.yaml", "temp_dir": "/tmp", "cpus": 2, "memory_b": 2000000000 }, "dep": { "dependency_name": "/path/to/dependency" } }Reviewing the viash work directory
To inspect the work directory and
params.jsonfile during execution:New feature: UNDEFINED and UNDEFINED_ITEM
This PR adds proper support for explicitly unsetting argument values:
Unsetting Single-Value Arguments
Pass the literal
UNDEFINED(unquoted) to set an argument to null:Result:
par["optional_arg"]isNone(Python),NULL(R),null(JS), or unset (Bash).Missing items in multi-value arguments
Use
UNDEFINED_ITEMin semicolon-separated values to represent null elements:./my_component --values "item1;UNDEFINED_ITEM;item3"Result:
par["values"]is["item1", None, "item3"](Python) or equivalent.Passing literal "UNDEFINED" string
Use nested quotes to pass "UNDEFINED" as a literal string:
Unsetting computational requirements
You can also pass UNDEFINED to unset
---cpusor---memorydefaults:Checklist